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FRAMEWORK FOR EVALUATING EDUCATIONAL SYSTEMIC INITIATIVES 



0.0 



Abstract 



T he inqjlementation of the Holistic Systemic Evaluation (HSE), a component of the Education 
Systemic Initiative’s strategic management is presented in this paper. The HSE provides 
general guidance for the implementation and continual inqirovement of the Education Systemic 
Initiative Reform (ESIR). Hence, holistic systemic evaluation plan should primarily be directed 
to those ofiBcials who are charged with the functional responsibility of conceiving, directing, and 
in5)lementing education activities within ESIR confines. Specifically, the implementation of the 
education systemic initiative plan; 

• Identified three leadership strategies to inqirove and guide education systemic initiative 
efforts. 

• Outlined the education agenda for the subsequent years through seven inqirovement 
initiatives. 

• Delineated the operating principles that are integral to the conduct of all the education 
initiative activities. 

• Defined the Evaluation Frameworks fi>r the Education Systemic Initiative Reform; the bases 
fi-om which education initiatives programs are organized, implemented, and evahiated. 

• Described the roles and responsibilities of the various departmental entities that carry out the 
Education Systemic Initiative Reforms. 
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FRAMEWORK FOR EVALUATING EDUCATIONAL SYSTEMIC INITIATIVES 



1.0 Introduction 

T he Education Systemic Initiative Project Director and/or Evaluator should understand the 
need and importance of an evaluation plan in education systemic initiative reform efforts. A 
linear evaluation process that incorporates the results of all subsystems and their components 
into eva;uation report phases where each phase reports on the results of the events and activities 
that happened during the reporting period is necessary. This method of reporting the progress of 
ESIR affords the management team the opportunity to strengthen and/or continue to focus upon 
the capacity building to achieve the overall project goals, streamline the results of the analyses, 
enforce policies that are deemed necessary during the evaluation reporting cycles, and improve 
upon the recommendations suggested in previous evaluation periods (Guskey, 1989, 1992; 
Heinich, Molenda, Russell, & Smaldino, 1996). 

The Education Systemic Initiative Plan should adopt the systems approach principles for the 
evaluation of the Education Systemic Initiative Reform (ESIR) changes in science, mathematics, 
engineering, and technology (SMET) education at all levels (i.e., K-12 and beyond). An 
evaluation research with a ‘ Holistic Evaluation Design, ’ has been chosen for this endeavor. 
Evaluation research is an applied research study that uses any of the several research methods 
designed to test the effectiveness or impacts of social changes or intervention (Guskey, 1988; 
Ikegulu & Barham, 2001; Mayers, 1999). The holistic evaluation design measures the degree of 
participation and shared responsibilities and/or ownership; moving away from compliance issues 
to accountability and utilization of allocated resources to maximize the convergence of available 
intellectual, faculty, personnel, and material resources. In a holistic sense, reform will occur 
when: (1) classroom teachers or instructors have the time, resources, permission, training, and 
space to experiment, try, fad, and/or achieve autonomy to sustain ownership; (2) students 
demonstrate the interest to learn and understand the classroom exercises to improve in their 
SMET education; and (3) parents, community, and/or business/umversity partners understand the 
need and importance of SMET education for minority and economically disadvantaged students 
in a growing population (Ikegulu, 1997, 1999a, 1999b). The Project/Program Evaluator and/or 
Director should also understand that uncertainties in socio-behavioral sciences and human 
populations. The evaluation philosophy should acknowledge that uncertainties in any component 
of the subsystem in both design and measurable attributes are evident and, as such, may and 
should not result in the failure of the super system (Guskey, 1992; Heinich, et al., 1996; Loucks, 
1977; Mayers, 1999; Scriven, 1975, 1971, 1968). Hence, the tendency for the evaluation plan to 
be deterministic deters the pragmatic holistic systemic approach paradigm. Therefore, 
predictions, shrinkage (or short-cuts), and generalization are not possible during the initial phase 
without sound methodological approaches (including research design) and sophisticated 
analytical procedures (Ikegulu, 1999a, 1999b; Scriven, 19971, 1975). Instead, realistic 
approaches that are prevalent in any socio-behavioral setting should guide the feasibility of the 
systemic reform to effect changes in the SMET environments within the ESIR confines. 
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1.1 Education Systemic Initiative Reform Goals: The Super System 

The Project/Program Evaluator and/or Director should be responsible for the planning, operation, 
implementation, and management of the ESIR project as described in the project’s proposal 
and/or its modification(s) and amendment(s). The overall mission of the ESIR project should be 
to develop educational systemic change approaches to produce significant increases in the 
number of students in the educational system who are literate and proficient in engineering, 
mathematics, sciences, and technology (within the public schools system); be qualified to pursue 
undergraduate programs (within the higher education system); and be able to compete in the 
national and global economy in SMET related professions. In order to accomplish this mission, 
the ESIR management team, spearheaded by the District's Superintendent or the institution’s 
President (as the Principal Investigator), the Co-Principal Investigators (the institutional 
Research Officer), the Project Director and/or Evaluator, and the SMET Supervisors (where 
applicable) should formulate the ESIR objectives. Collectively, these goals are the system’s 
goals. Sample system’s objectives include: 

• To enact policies and practices that will increase PreK - 16 students' mathematics test scores 
on the appropriate standardized tests by at least 5% within each school and/or academic year. 

• To increase the district's or institution’s enrollment in Algebra, Biology, Physics, Chemistry, 
Computer Science, and/or Engineering Graphics by 50% by the end of the ESIR project. 

• To double the number of students (directly involved in the ESIR Project) who will enroll in 
and successfully complete the SMET courses in Algebra, Biology, Chemistry I, Physics, Pre- 
Calculus, Calculus, Computer Science, Engineering Graphics, etc. by the end of the ESIR. 

• To increase the number of students (directly involved in the Project) fi-om the baseline year 
and, successfully complete the gate keeping course (with a grade of C or better) by 100% 
before the final year of the ESIR Project. 

• To double the number of SMET high school graduate who are college-bound (and the 
number of SMET graduate-level student in the Project) by the year-end of the project. 

• To establish and implement articulated standards-based SMET curricula throughout the 
district and/or the higher education community. 

• To implement an articulated standards-based curricula, instructional, and assessment 
programs in PreK- 16 and beyond in SMET education. 

In response to the state and national demands and, with the support fi-om the funding agency, the 
ESIR management team should place emphases on mathematics, science, and technology 
education towards the enhancement of teaching strategies to include the creation of stimulating 
and challenging learning environments; and to focus upon on-going assessment criteria and 
standards-based curricula implementation in all the ESIR classrooms. To achieve these 
objectives, the ESIR’s management team should focus on the following systemic changes: 
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• a focus on high SMET expectations and a high level of competence of all students; 

• a strengthening of professional competency of all faculty and staff; 

• an integration of SMET into a standards-based curricula from PreK-16 and beyond; 

• the development of quality assessment materials that are aligned with state, national, and 
professional frameworks and, are also standards-based, inquiry-dependent instructional 
models. 

1.1.1 Components of the Subsystems: Synergy within the Super System 

Figure 1-A is a schematic illustration of the interrelationships among the ESIR systems. This 
figure also portrays the logical flow of information within the super system and its components. 
The terminal objectives are embedded within the enabling objectives (ENAB_OBJ I to VI). 
Collectively, the ESIR objectives are the system’s goals; the super system. The subsystems are 
the six enabling objectives and the micro subsystems are the terminal performance objectives 
with measurable attributes (See Figure 2). 




Figure 1-A 

A Linear Evaluation Process Using the Holistic Systemic Evaluation Paradigm 



Figure 1-B is a cyclical illustration of the links between the subsystems and the super system, as 
well as among the super system, the subsystems, and the evaluation process. This figure also 
portrays the timeline for the completion of the activities within the each evaluation phase. 
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Figure 1-B. A Cyclical Holistic Evaluation Process Using the Systemic Evaluation Paradigm 
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2.0 Education Systemic Initiative Reform Agenda and Strategy 

T he Education Systemic Initiative Reform (ESIR) is broad and conqjrehensive. Seven specific 
reform inqjrovement and implementation initiatives have been identified to guide the project 
and/or program(s), hiunan and financial resources, and activities during the life span of the 
project (Initial to Final) school years. These initiatives guide the existing efforts as a school 
district or higher education institution and, serve as priority areas for capacity building, initiation 
of new activities, and establishment of sustainable infi-astructures. It is a conprehensive list that 
will be carried out by the school district as a whole through the Project Director/EvaKiator, the 
district’s administrative stafl^ and the canpus-based administrators. The seven reform 
inprovement and implementation initiatives are: 

1. Focus and Coordinate Canpus-based Efforts. 

2. Enhance Curricuhun/Instructional and Assessment Products and Dissemination. 

3. Inprove Education Systemic Initiative Reform Integration and Coordination. 

4. Facilitate Research within the School Districts and Higher Education Community. 

5. Support Pre-service Education through StafBTrofessional Development. 

6. Coordinate Informal Education through Community and Adult Education Programs. 

7. Implement ESIR’s Conprehensive Data Collection and Evaluation System. 

2.1 Focus and Coordinate Campus-based Efforts 

The Education Systemic Initiative Reform is in one sense composed of all the campuses within 
the school district, as well as the SMET departments in higher education. The reform of 
mathematics, science, technology, and engineering in PreK-16 schools should be inextricably 
linked to the state and national higher education systems, as well as be aligned with the local, 
state, and national agendas for educational development. Central to the campus-based focus is 
the need for ESIR management to understand the campus education agendas and place emphases 
on coordinating its assets in a given campus toward meeting the ESIR related needs of that 
campus. By continuing existing and establishing new alliances, ESIR management should seek 
to connect the Principal Investigator, College Deans or Campus Principals, SMET supervisors or 
Department Heads, and business and other university partners and leadership to determine how 
these assets may best be utilized within the campus. Major actions to be taken include: 

1. Campus principals and/or College Deans will establish relationships with the 

District's SMET supervisors and/or Department Heads to understand, develop, and 
coordinate ESIR’s educational efforts in each campus and/or department. 

2. Campus-based curriculum coordinators. Department Heads, program/grant managers 
or facilitators, and other ESIR education- funded directors/coordinators will establish 
direct linkages with the Principal investigator (through the Project Director and/or 
Evaluator) and contribute to the coordination and delivery of the ESIR’s education 
reform agendas within a given campus/department. 
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3. The Principal Investigator (through the SMET Supervisors and Directors) will 
continue to develop linkages and support local, state, and/or national organizations 
that assist ESIR’s Project Director and/or Evaluator in achieving the canqjus-based 
and/or departmental initiative. Such support organizations include the Clergy-in- 
Schools, Foimdations, the NASA, the NSF, the National Alliance of State Science 
and Mathematics Coalitions, the Association of State Supervisors of Mathematics, the 
Coimcil of State Science Supervisors, and other relevant education associations (e.g., 
NCTM, NCES, ITEA, etc.). 

2.2 Enhance Curricuhim/Instructional and Assessment Products and Dissemination 

One of the ESIR’s missions is to produce new data, images, and information that may be 
effectively included (and/or published) in professional journals, textbooks, curricula, and 
supplementary instructional products. Working with professional education associations, 
state/local and national education authorities, universities, private enterprise, and other 
organizations, we will collaborate to develop curriculum guides, instructional aids, and 
assessment products consistent with the national curriculum and assessment standards and/or 
state or local curriculum frameworks. These products will be developed in multiple formats with 
emphasis on innovative applications of educational technology and interactive strategies. Major 
actions to be taken include: 

1. Develop and distribute a handbook outlining the ESIR's protocols for the 
development, review, field-testing, and distribution of instructional materials and 
assessment practices. 

2. Develop a set of PreK-16 instructional products, in cooperation with the state and 
national curricula standards to support the district’s and departmental locally 
established curricula in mathematics, science, technology, and engineering. 

3. Develop a set of PreK-16 district-wide and institutional assessments that are in 
alignment with state and national frameworks. 

4. Develop and distribute the ESIR’s Evaluation Plan, including plans for Professional 
Development, Quality Control, and Data Collection, Disaggregation/Dissemination, 
and Utilization. 

2.3 Improve Education Systemic Initiative Reform Integration and Coordination 

The project’s Education Systemic Initiative Reform consists of many parts, which, when 
working together as a whole, can make significant and positive contributions to the education 
community. The Implementation Plan should be designed to ensure that the research design, 
coordination, and implementation of ESIR's numerous educational projects, programs, and 
activities achieve this vision of a coherent and unified educational systemic initiative reforms to 
effect changes that will impact the overall image of the education community, as well as the 
quality of education of all students. 
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The educational function exenqjlifies a higher degree of integration and coordination that has 
ever been ejq>erienced in the 50-year history of any ESIR. However, work remains to improve 
the integration and coordination of these efforts. Inqjroved program integration and coordination 
have to be acconqjlished at three distinct and interrelated levels: (1) Institution/District (2) 
Can 5 >us/Department and (3) Program/Project and the community. Major actions in this category 
include: 

1. At the Institutional/District’s Level: T he Institutional President/Superintendent (as the 
Principal Investigator) provides the focus, policy, and general direction for Project’s 
Education Systemic Initiative Reform. It is incumbent upon the ESIR's management 
team and the institutional/district’s administrative staff, the Director of Research, 
Evaluation, and Planning, Director of StafFProfessional Development, Coordinator of 
Gifted/Talented and At-risk, Director of Communications, Director of Information 
Services and Technology, Director of Special/Developmental Education, SMET 
Supervisors, Department Heads, and the Project Director and/or Evaluator to 
formulate strategic education functions to provide the leadership and strategies for a 
single, unified Education Systemic Initiative Reform agendas. 

2. At the Campus/Departmental Level: The campus-based and/or departmental 
administrators and education staff (College Deans, Department Heads, instructors, 
classroom teachers, librarians, counselors, etc.) should be responsible for 
implementing the overall project’s Education Systemic Initiative Reform. It is 
incumbent upon the campus principals and their administrative staff to establish and 
maintain organizational mechanisms to provide District-wide integration and 
coordination of programs, projects, and activities. Additionally, the school principals, 
administrative staff, and curriculum coordinators or department heads must identify 
linkages among the campus programs, projects, and activities for the ESIR. 

3. At the Program/Proiect Level: The directors, coordinators, and/or managers of 
individual programs/projects or subprograms/subprojects must identify linkages to 
other programs/projects and revise plans to ensure that these linkages are 
incorporated into and aligned with each of the campus-based and District’s activities 
(departmental and institutional). These personnel must also ensure that the 
program/project or subprogram/subproject participants are made aware of related 
activities from which they may benefit. 

2.4 Facilitate ESIR Research within the School district and Higher Education Community 

Research relevant to the ESIR’s three strategic improvements of the integration and coordination 
efforts is carried out primarily throughout the district and/or institution. However, the highly 
complex/complicated ones should be being conducted by the Project Evaluator. The Project 
Evaluator in conjunction with higher education community or universities will implement 
research projects that should be focused on higher education systemic initiative reforms. The 
goal is to streamline and focus these latter efforts so that they strongly support the district’s 
research objectives as determined by the ESIR management team and approved by the Principal 
Investigator. Major actions in these areas include: 
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1. Merge similar programs/projects so that the best attributes of all are retained and 
enhanced. 

2. Closely align research-related programs/projects to the district’s, institutional, 
campus-based, and departmental evaluative research agendas. Specifically, realign 
the Project's overall goals to meet ESIR’s research objectives and the canq)us-based 
research-related infi^tructural priorities. 

3. Collect, evaluate, and disseminate the most in 5 )ortant research products (e.g., 
publications, awards, and technology transfer) so that the district/institutional and 
campus personnel recognize these efforts as relevant and in^ortant research 
contributions. 

2.5 Support Pre-Service Education through StafBTrofessional Development 

Various national reports indicate that there will be a shortage of PreK-16 science, engineering, 
mathematics, and technology teachers over the next ten years. Concomitantly, institutions 
responsible for training the next generation of public school teachers are aligning their pre- 
service academic programs with new certification requirements and public policy expectations. 
While ESIR's existing in-service programs (in the form of district-wide and campus-based staff 
or professional development workshops, seminars, institutes, etc.) need to continue at their 
present level and pace, it is important for the ESIR management team to focus on new 
opportunities to support initiatives in the pre-service area. The ESIR Project's significant 
investments in research and development with institutions of higher education will provide a 
unique asset in identifying such opportunities. Major actions in this area should focus on: 

1. Continue to refine and support district's Summer Institutes. Continue to offer stipends 
to attract qualified and certified mathematics teachers. 

2. Continue to support the instructional faculty/staff through the tuition-reimbursement 
at the local and other institutions of higher education’s pre-service activities that seek 
the collaboration of mathematics, science, engineering technology, and education 
departments in preparing the next generation of teachers. 

3. Implement a two- to three-year inquiry-based and hands-on pre-service training in 
science and mathematics initiatives through the higher education community. The 
two- to three-year initiatives will model an approach that may be replicated in other 
regions or, serve as an ESIR’s motif for content-specific pre-service training model. 

4. Encourage ESIR-sponsored researchers at the district and universities to collaborate 
with pre-service education faculty to contribute to teacher preparation. 
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2.6 Coordinate Informal Education through Community and Adult Education: 

Informal Education Programs 

Educational enrichment should be provided to the community through the Adult Education and 
Career Centers. Exploratorium, Planetarium, Museums, Science and Technology Centers, field 
trips, and similar nonprofit education organizations should be called upon to support the informal 
education community and provide significant educational activities for learners at all ages and 
levels of education. Most of these organizations are major community, regional, state, or national 
resources for science, mathematics, technology, and engineering technology education. In 
addition, the informal education community has a tradition of presenting educational experiences 
using pedagogic principles, inquiry-based, and hands-on approaches that are well aligned with 
the National Science Education Standards (NSEC) and the National Council of Teachers of 
Mathematics (NCTM) fi’ameworks. The ESIR management team should work with and support 
these organizations. Major actions to be considered in this area include: 

1 . Define an ESIR mechanism for working with the informal science, mathematics, 
technology, and engineering technology education community. 

2. Develop planetarium and galaxy programs and projects for local, state, and national 
distribution. Place students’ exhibits on campus and local libraries, district’s and/or 
institutional archives, and/or students’ portfolios. 

3. In collaboration with the district, develop and implement on-going campus-based 
programs to support the informal education community. 

4. Continue to strengthen the ESIR and funding agency’s education aUiance. 

5. Support Mathematics Clubs, National Science and Technology Week, NASA week, 
Space Day, Space Week, and National Engineers Week. 

2.7 Implement ESIR's Comprehensive Data Collection and Evaluation System 

In carrying out the equal education opportunity for all, the ESIR management team should strive 
to involve all students, community, and educators as both participants and partners in quality 
education. In conforming to the Federal Government Performance and Results Act of 1993, the 
ESIR Project should be committed to evaluating the performance of its programs/projects and 
activities in order to report to the stakeholders, the community at-large, and the funding agency 
and, to provide for continual improvement of such involvement of the educational community in 
its missions, research endeavors, development, and achievements. To that end, the ESIR’s 
management team should develop a Database Management and Evaluation System (The DMES), 
an on-line and Internet-based system for data entry and collection fi’om participants and program 
managers. The DMES is a data collection tool that includes raw data in SPSS and Excel formats; 
briefing and statistical presentation materials to be used for analysis and reporting; protocols for 
follow-up studies; and raw data to be analyzed for future documentation and presentation. Major 
actions include: 
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1. Continue systematic iii 5 )lementation of the evaluation phases. 

2. Continue to in^rove upon the DMES c^ability to enhance its accuracy, current 
features, dependability, options, reliability, and robtistness. 

3. Strive toward a folly operational ESIR Database Management and Evaluation System 
that collects all ESIR Project’s data. 

4. Continue to increase the district's and/or institutional \jse of the DMES for 
educationally related and relevant activities and programs where feasible. 

5. Increase techniques and processes for conducting follow-up studies. 

6. Generate research topics, questions, and hypotheses germane to the field of systemic 
initiative reforms, evaluative studies, and education in general. 



3.0 Evaluation Framework for the Education Systemic Initiative Reforms 

T he ESIR Evaluation fiiumework (See Figure 2) is a model that was established to serve as a 
guide for the m^)lementation and evaluation of all the Education Systemic Initiative Reform 
efforts. The fi-amework was first proposed in fone 1998. In September 1998, the proposed 
fi-amework was presented to the ESIR management team. This meeting resulted in the overall 
acceptance of the fi-amework and, the general consensus on its adoption and the establishment of 
the nodes that were aligned with the goals for each iiiq)lementation ^proach. From that time 
imtil now, the fi-amework has been refined and updated; reflecting directions defined in the 
Education Systemic Initiative Reform Agendas and Strategies. 

The Evaluation fi-amework for the Education Systemic Initiative Reform provides a pictorial 
representation of the project’s systemic initiative reforms. It serves not only as a guidance tool, 
but also as an analytical mec’nanism, in conjunction with Figure 3, to evaluate the 
coit 5 )rehensiveness of the ESIR’s outreach to the education community. The fi-amework depicts 
the integration of the three components of all systemic initiative reforms and agendas, programs, 
and activities. These three conq)onents are: 

3.1 The Enabling Objectives: The ESIR Mission Statement 

The fondamental component of any ESIR education activity is the content or knowledge derived 
fi-om the funding agency’s mission. There are six of these goals. At the institutional/District’s 
level, the knowledge is the outcome of the ESIR mission as defined by the three functional and 
strategic integration and coordination of the systemic initiative reform efforts within the 
District/institution level(s), as well as at the campus/departmental and various programs and 
activities. The knowledge derived fi-om the ESIR funding agency’s objectives is the content, and 
thus, the foundation for all of the ESIR's education activities. The role of ESIR's systemic 
initiative reform is to add value by translating this content to meet the ESIR’s clients' needs. 
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3.2 The Client 

The ESIR's client is the formal and informal education community. For the purpose of the 
framework, the formal education community is divided into the following levels based on grade: 
(1) the public school system (PreK-12) broken down into (a) PreK-5 or the elementary school 
system, (b) 6-8 or the middle school system, and (c) 9-12 or the high school system; (2) 
Community and Adult Education (CAE) offered through the Adult Education and Career Centers 
in the form of vocational/technical education and on-going technology seminars, workshops, 
and/or institutes; and (3) Undergraduate and Graduate Education (UGE) or the college- bound 
public school graduates in SMET academic programs and careers. 

At the PreK-12 levels, the content (or knowledge) derived from the ESIR mission is tailored to 
meet the clients’ needs and is guided by curriculum standards and assessment practices for 
science, mathematics, technology, and engineering at the local, state, and national levels. At the 
postsecondary levels, the clients and the educational opportunities offered to them are directly 
involved in, and in support of ESIR's mission. The informal education community targets both 
the PreK-12 and postsecondary levels and, includes mathematics, science, and technology 
centers, museums, planetariums, and other nonprofit education organizations. 

The education of the clients is both the beginning and the ending point for all the ESIR’s 
education activities. It is the clients' education agenda that serves as the starting point in defining 
an educational project or activity. Furthermore, evaluation of the clients’ success validates 
ESIR's education activities to determine whether the district(s) and the higher education 
institution(s) are contributing to its clients’ educational excellence. 

3.3 Educational Programs and Support Services 

Six categories comprise the ESIR’s systemic initiative reforms and define the way in which the 
ESIR’s six Enabling Objectives (ENAB_OBJ) are delivered to the formal and informal education 
community. The following is a summary of these six programs and support services and 
activities, including the goals, objectives, and specific expectations or actions. 

3.3.1 StaflFProfessional Development 

The primary objective here is to use the ESIR’s mission, facilities, human resources, and 
programs to provide exposure and experiences to educators and faculty, to support the 
enhancement of knowledge and skills, and to provide access to ESIR information in science, 
mathematics, technology, engineering, and geography. The ESIR’s management team uses its 
facilities, human resources, and programs to build its infrastructures and, involve educators and 
faculty to advance their knowledge and skills. These programs and support services and activities 
are designed to provide professional development experiences for PreK-16 instructional staff 
and educators, non-instructional staff, and/or higher education faculty that are involved in pre- 
service education. Additionally, some programs are designed to provide research opportunities. 
The educators and faculty (a) participate in ESIR projects, programs, and research and 
development activities, (b) apply methods for integrating these resources into their teaching, and 
(c) are informed about available ESIR resources. The targeted objectives include: 
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1 To provide ESIR’s mission-based programs that will introduce the application of science, 
mathematics, engineering, and technology for use in student learning activities. 

2. To provide educators with a wider range of alternatives using scientific inquiry based on 
the ESIR's missions and agendas. 

3. To encourage a " multiplier " effect to expand the benefits of the in-service programs 
beyond participants to include additional educators. That is, to make good use of the Lead 
Teacher Program. 

4. To provide access to and promote utilization of ESIR-related materials and information 
resources. 

5. To increase the participation of underserved and under-utilized individuals and groups. 

6. To facilitate collaborations between the fiiculty/staff of teacher preparation departments 
and the fiiculty/staff of scientific and technical departments to develop irmovative 
approaches to teacher preparation. 

We expect to accomplish the following activities: 

1. Model inquiry-based science investigations and meaningful mathematics problem solving 
by engaging educators and teachers in the kinds of learning they are expected to practice 
with their students. 

2. Expand follow-up and networking opportunities for the BISD femily (teachers, parents, 
business/university partners, etc.) with enhancement programs through the use of 
electronic means, conferences, and training sessions. 

3. Expand the scope of educator enhancement programs to include workshops at each. 
Define and execute activities that target pre-service systemic initiative reforms. 

4. Provide education experiences for educators in the effective application of educational 
technologies (such as computers, telecommunications, videoconferencing, and CD-ROM) 
to present content and pedagogy. 

3.3.2 Standards-based Curriculum and Assessment 

The purpose here is to develop, utilize, and disseminate science, mathematics, and technology 
instructional materials based on ESIR's unique mission and results, and to support the 
development of higher education curricula and assessment. The ESIR develops, utilizes, and 
disseminates science, mathematics, and technology assessment, curriculum, and instructional 
materials based on ESIR's unique mission. Because education is primarily a state and local issue, 
we seek to broadly understand common curricula standards and topics, assessment practices and 
standards, collaborate with outside education experts, and work with the ESIR’s content experts 
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to translate the enabling objectives and/or goals into supplementary instructional sequences and 

assessment packages. The following are some of the targeted objectives: 

1. To support activities that fecUitate an exchange of information among higher education 
curriculum developers and instructional designers. 

2. To develop science, mathematics, and technology instructional materials that support 
curricula that are aligned with national standards and state frameworks and, are based on 
ESIR's unique mission and results. 

3. To review and revise existing ESIR’s instructional materials to align with the national 
standards and state frameworks and update content based on ESIR's unique mission and 
results. 

4. To facilitate the use of the ESIR’s instructional materials through educator and in-service 
training programs. 

5. To disseminate the ESIR’s instructional materials to educators by fully utilizing Project's 
distribution system for educators at the canq)us and/or departmental levels. 

The following ejqjectations are in order: 

1. Expand the use of the ESIR’s curriculum support products in educator and in-service 
training programs. 

2. Inq)rove the development process for educational products by creating and disseminating 
a handbook to provide guidance to individuals and organizations developing the ESIR’s 
curriculum support products. 

3. Replace outdated ESIR’s educational products with new versions that support the 
national education standards and state frameworks and, that contain the most accurate 
and current contents derived from the ESIR’s goals. 

4. Improve communications among the district’s/institutional Information Services and 
Technology (1ST), campiis/departmental administrators, SMET supervisors, and canq)us- 
based curriculum coordinators. Provide quality training on ESIR’s content and increase 
the use of ESIR’s assessment, curriculum, and instructional products. 

5. Through the Project’s systemic initiatives, ensure that campus-based IST's become part 
of the systemic initiatives on each canqjus and review the canq)us-based IST's mission to 
ensure their alignment with the overall ESIR's holistic educational objectives. 

6. Where ^>propriate and feasible, ensure that all assessment, curriculum, and instructional 
support products are available electronically. 
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3.3.3 Business and University Partners 

The main focus here is to use the ESIR's unique assets to support local, regional, state, and 
national science, mathematics, technology, and engineering education systemic reform change 
efforts through collaboration with internal and external stakeholders. As the United States 
continues to reform science, mathematics, technology, and engineering instruction in its PreK-16 
and beyond schools, the ESIR management team has placed emphases on coordinating all of the 
ESIR’s assets in a given campus toward assisting in meeting its goals for improvement of the 
school’s system of education. By establishing a variety of partnerships, the ESIR’s management 
team seeks to convene the Principal Investigator, SMET supervisors, campus principals, lead 
teachers, and commercial contractors with the local and state education leadership to determine 
how these assets may best be utilized. The targeted objectives in this area include: 

1. To coordinate planning among the ESLR education initiatives to ensure alignment with 
and support of standards-led systemic improvement initiatives of the states. 

2. To redirect existing systemic initiative reforms, and to ensure that new initiatives address 
local and state needs together with their unique education and economic development 
efforts. 

3. To support standards-based science, mathematics, technology, and engineering education 
change by aligning ESIR Project’s educational programs and products with the 
national/state standards. 

4. To expand interactions with external stakeholders in the systemic improvement of 
education change. 

The following accomplishments are expected: 

1. Provide professional development and support on standards-based education initiatives to 
all the ESIR's internal education community. 

2. Review all the existing ESIR’s education initiatives to ensure their alignment with the 
national vision and philosophy for standards-based systemic reform. 

3. Design new programs or redesign existing programs to ensure that all the ESIR’s efforts 
align with the science, mathematics, technology, and engineering education standards and 
support the needs of those engaged in the implementation of standards-based science, 
mathematics, technology, and engineering education at the state and local levels. 

4. Leverage the use of the ESIR’s programs and resources by expanding ESIR management 
team’s interactions and cooperation with all stakeholders involved in national and state 
systemic initiatives. 

5. Through the campuses and departments, develop and implement a plan that supports the 
needs of individual school and/or Project site. 
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3.3.4 Student SutJPOrt 

The goal here is to utilize the ESIR’s mission, facilities, human resources, and programs to 
provide information, experiences, and research opportunities for students at all levels to support 
the enhancement of knowledge and skills in the areas of science, mathematics, and technology. 
Student support programs are intended to involve students in the intrinsically interesting and 
informative BISD-CPMSA and NSF common goals. The objectives in this area are: 



1. To provide experiences and information that are designed to promote students' interest in 
SMET. 

2. To provide exposure to research and/or research experiences and activities to promote 
SMET academic and career awareness. 

3. To provide support to the science and technology workforce pipeline by including greater 
participation of individuals who are under-represented in SMET. 

4. To increase the number of student support opportunities through partnerships and small 
business cooperation and collaboration. 



We expect to accomplish the following: 

1 . Develop and maintain electronically disseminated communication of all the ESIR- 
sponsored student opportunities and career information. 

2. Coordinate ESIR's student program efforts and ensure the progression of student 
programs. 

3. Develop and field-test models for involvement strategies in all student programs. 
3.3.5 Classroom Technology Integration 



The primary goal here is to research and develop products and services that facilitate the 
application of technology to enhance the educational process for formal and informal education 
and lifelong learning. The ESIR's educational technology program should support at least two of 
the four components of the National Educational Technology Initiative. The ESIR Project should 
address the development of innovative learning tools and strategies (that are usable in the 
classrooms), as well as teacher training opportunities for the information providers (Guskey, 
1988, 1989, 1992; Loucks, 1977). The objectives in this instance are: 

1. To produce technology-based teaching tools and strategies that are grounded in or 
derived from the ESIR’s mission. 

2. To use emerging technologies for, and apply existing technologies to, public and higher 
education programs and curricula. 
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3. To utilize technology to facilitate communication within the educational community. 

4. To involve educators, stafl^ students, and the community at-large in the ESIR’s missions 
through innovative uses of technologies. 

5. To conduct empirical studies in new teaching and learning practices that is made possible 
through the ESIR’s mission-derived technology. 

The expectations in this category are as follows: 

1. Provide technology training and support for the persons involved in the operation of the 
Information Services and Technology network and other programs as necessary. 

2. Inclement a coordinated electronic dissemination system that ensures that all ESIR 
Project’s education activities and products are available through ^propriate networking 
technologies (that is, the Internet and satellite or cable television). 

3. Demonstrate ESIR's educational technology resources at professional development 
conferences and other relevant venues. 

4. Develop innovative learning tools and technologies that are integrated with curriculum 
support and teacher enhancement activities. 

5. Develop, irr 5 )lement, and evaluate distance education and virtual mentoring projects. 
3.3.6 Research-based Studies 



The goal in this area is to involve the education community, particularly higher education, in the 
ESIR’s programs that contribute to the development of new knowledge in support of its 
missions, and to utilize the talent and resources of the higher education community. Research 
and development activities occur primarily, though not exclusively, at the graduate level and 
involve students (undergraduate and graduate) and faculty who make substantive contributions to 
ESIR's mission. In addition to directly supporting the Project’s academic programs, these 
activities promote the development of new collaborations with the academic community; 
significantly enrich pre-service and graduate education, as well as research opportunities 
(Ikegulu, 1997, 1999a, 199b, 2001; Ikegulu & Barham, 2001; Loucks, 1977; Mayers, 1999; 
Ogunyemi & Ikegulu, 1998; Scriven, 1975). The other goals are: 
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Figure 3: A Hierarchical Model of Holistic Systemic Evaluation 
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1. To provide administrative infrastructure to support ESIR-sponsored programs for the 
academic research community. 

2. To communicate opportunities for the ESIR-sponsored research and changes in ESIR 
Project’s research policy to the academic community. 

3. To act as a liaison between the higher education community and the district. 

5. To establish and maintain communication among the district’s staff, the ESIR’s 
management tern, the ESIR-sponsored university programs, and the can^us-based 
staff, faculty, students, and teachers. 

5. To promote sponsorships among the business and industrial community with ESIR’s 
management team and the ESIR-sponsored research programs. 

6. To encourage the transfer of ESIR-funded technology development to the commercial 
sector and to the American taxpayer. 

Major expectations include: 

1 . Develop, in consultation with relevant stakeholders, an mq)lementation plan for all the 
ESIR’s research programs sponsored by the fimding agency. 

2. Produce and maintain an electronically disseminated commumcation system of the ESIR- 
sponsored research opportunities, which provides a single point of information for 
individuals and organizations seeking to participate in systemic initiative research 
activities. 

3. Track, evaluate, and disseminate the results of research programs sponsored by the 
funding agency through the ESIR’s ‘ Holistic Education Evaluation System. 

4. Increase the visibility of university programs, particularly within the ESIR confines, and 
the unique role these programs play in fecilitating collaborations with the academic 
community. 

3.4 The Holistic Systemic Evaluation: Analytical Framework 

The framework for the ESIR evaluation plan is as shown in Figure 2. The analytical framework 
for the evaluation studies in ESIR is as depicted in Figure 3. A major chaUenge to full-scale 
evaluation of any systemic initiative reform project has been the ability to articulate and integrate 
the subsystems into a coherent functional umt for the betterment of the overall student and 
feculty/staff population in the district and institution, the community, and the project sponsor(s). 
The evaluation plan adopted for the educational Project’s systemic initiative reform changes in 
SMET is a ‘Holistic Evaluation Design. ’ This is a global systems approach paradigm that 
evaluates the micro subsystems first, uses the results from these micro subsystems to assess the 
effectiveness of the subsystem, and the evaluation results from the subsystems to judge the 
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ESIR’s management team should endeavor to use its facilities, human resources, and programs 
to maximize its capacity building, build and sustain its established infrastructures, and involve 
educators and faculty/staff to advance their knowledge and skills. These programs and support 
services and activities should be designed to provide professional development experiences for 
PreK-16 instructional staff and educators, non- instructional stafl^ and/or higher education feculty 
who are involved in- and pre-service teacher education programs. Additionally, some programs 
should be designed to provide research opportunities that will enhance the overall image of the 
ESIR Project within the community, campuses, departments, and/or Project sites. The educators 
and faculty (a) participate in ESIR Projects, programs, and research and development activities, 
(b) apply methods for integrating these resources into their teaching, and (c) are informed about 
available ESIR resources. 

4.0 Conclusion 

T he Holistic Systemic Evaluation, one component of the ESIR’s strategic management 

system, provides general guidance for the in5)lementation and continual in5)rovement of the 
Education Systemic Initiative Reform throughout the life of the Project. Hence, the ESIR 
Evaluation Plan should primarily be directed to those officials who are charged with the 
functional responsibility of conceiving, directing, and inqjlementing education activities within 
the Project’s site(s). Specifically, the inqjlementation plan: 

• Proposes a holistic systemic evaluation paradigm. 

• Identifies three leadership strategies to in 5 )rove and guide our efforts. 

• Formulates an evaluation framework that links the six enabling objectives to the clients and 
the educational programs and support services. 

• Proposes an analytical evaluation framework that will guide the conduct, data collection, and 
analytical procedures throughout the tenure of the project. 

• Outlines the ESIR’s education agendas through seven inqjrovement initiatives. 

• Delineates the operating principles integral to the conduct of all the ESIR’s education 
activities. 

• Defines the Evaluation Framework for the Education Systemic Initiative Reform, the basis 
from which the district-wide and institutional, campus-based and departments, and programs 
are organized, in^lemented, and evaluated. 

• Describes the roles and responsibilities of the various departmental entities that carry out the 
Education Systemic Initiative Reform activities. 




25 



Evaluation Framework 23 



5.0 



REFERENCE 



Guskey, T. R. (1992). What does it mean to be “research-based?” The Developer, p. 5 . 

Guskey, T. R. (1989). Attitude and perceptual change in teachers. International Journal of 
Educational Research. 1314). 439-453. 

Guskey, T. R. (1988). Teacher efficacy, self-concept, and attitudes toward the 
implementation of instructional innovation. Teaching and Teacher Education. 4(1). 63-69. 

Heinich, R., Molenda, M., Russell, J. D., «fe Smaldino, S. E. (1996). Instructional media 
and technologies for learning (5*** ed.l . Englewood Cliffs, NJ; Prentice-Hall. 

Ikegulu, T. N. (1999b). Globalization and Co-integration of Universal Education . ERIC 
Clearinghouse and Journal of Resources in Education (ERIC Reproduction Service No. ED 451 
823). 

Ikegulu, T. N. (1999a). Multicultural and Globalized Education: International Students’ 
Perspective . ERIC Clearinghouse and Journal of Resources in Education (ERIC Reproduction 
Service No. ED 442 417). 

Ikegulu, T. N. (1997). Effectiveness of Mediated Instructional Strategies and Learning 
Styles in Muticulturallv Linguistic Environments: Implications for Developmental Educators. 

ERIC Clearinghouse and Journal of Resources in Education (ERIC Document Reproduction 
Service No. ED 409 757). 

Ikegulu, T. N. (1997). Effects of Institutional Experience and Family Support on Post- 
Developmental Students’ Academic Performance . ERIC Clearinghouse and Journal of Resources 
in Education (ERIC Document Reproduction Service No. ED 430 042). 

Ikegulu, T. N. & Barham W. A. (2001). Gender Roles, Final Course Grades, and Faculty 
Evaluation. Journal of Research and Teaching in Developmental Education. 17(2). 53-65. 

Ikegulu, T. N. & Barham, W. A. (1997a). Students’ Intentional Persistence As A Web Of 
Causal Factors: A Preliminary Study II. ERIC Clearinghouse and Journal of Resources in 
Education (ERIC Document Reproduction Service No. ED 428 643). 

Ikegulu, T. N. & Barham, W. A. (1997b). Students’ Intentional Persistence As A Web Of 
Causal Factors: A Preliminary Study I. ERIC Clearinghouse and Journal of Resources in 
Education (ERIC Document Reproduction Service No. ED 428 642). 

Loucks, S. F. (1977). Levels of use of the innovation; The conceptualization and 
measurement of a variable usefUl for assessing innovation implementation by individuals. (ERIC 
Document Reproduction Service No. ED 137 947). 

Mayers, D. P. (1999). Measuring instructional practice: Can policymakers trust survey 
data? Educational Evaluation and Policy Analysis. 2Un. 29-45. 

Ogimyemi, A. O. & Ikegulu, T. N. (1998). The differential effects of students’ gender 
and expected course grades on faculty evaluation. Louisiana Educational Research Journal. 

13(21. 23-50. 

Scriven, M. (1975). Causation as explanation. NOUS. 9. 3-16. 

Scriven, M. (1971). The logic of cause. Theory of Decision. 2. 49-66. 

Scriven, M. (1968). In defense of all causes. Issues in Criminology. 79-81. 




U.S. Department of Education 

Office of Educational Research and improvement (OERI) 
National Library of Education (NLE) 

Educational Resources Information Center (ERIC) 

REPRODUCTION RELEASE 

(Specific Document) 




TM034705 

i 



I. DOCUMENT IDENTIFICATION: 



Title: 

FRAMEM)RK R)R EVALUATINS EDUCATiCWAL SiSEMC INITIAUVES 



Author(s}: ^ Dr,. — T. Nelson IkeRul u. Research Assnriafp 

Corporate Source* . f ’-u Publication Date: 

ge '^Education. Depart of EducAtional Leadership November 5. 1999 

, Erambi i ng — Ei-airo iin-i irQi-pn~ <-y 

II. REPRODUCTION RELEASE: 

In order to disseminate as widely as possible timely and significant materials of Interest to the educational community, documents announced in the 
monthly abstract Journal of the ERIC system, Rtasotrroes in Education (RJE), are usually made available to users in microfiche, reproduced paper copy, 
and electronic media, and sold through the ERIC Document Reproduction Service (EORS), Credit Is given to the source of each document, and, if 
reproduction release is granted, one of the following notices te affixed to the document. 

If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following three options and sign at the bottom 
of the page. 




L®Vdl 1 Level 2A Lovol 2B 



> 1 I 

E] □ □ 

Chedc h®r® ^ retearo. perraittbig ie 3 }roducaon dieek hare for Level 2A reteaee. pwirttUlng raproducUoft Check hero tor Level 2B lelease. pemilttlno 

end dlesenmaoon n mlcroflclie or oOter ERIC anchlval and diseenOiaiSon Ih mlcroflche end In elec^onlc me<Qa reproduction and dlsecntneCon En microlicne only 

me«a (e-Q.. electronic) ond paper copy, tor eric archlvei collection subsen'bere on?y 

Documents wG be proocssod e» indicated pooveded reproduction <|ual]|y permits. 

If permission to reproduce Is granted, but no boor la checked, documents will be processed ai Lefvei i . 





/ hereby grant to the Educationei f^sourtes infemetion Center (ERiC) nenexdusive permission to leproduce and disseminate this document 
as indicated abova. Reprodueddh from the ERtC mtcrofiche or efectronio media by persons other than ERiC employees end its system 
contractors reQuires permission fiom toe oopyiight hoider Exception is made for non~prodt reprodudion by tibraries artd other seiyice agencies 
to satisfy infyrmafidn needs of educators in fesponse to effsenste inquiries. 


Sign 

here,-* 




l^nied Nmio/PoaBloo/TIttep 

Vrr ‘7'r a/' IK^ uL. 


please 


. LA- 

^ f ’ ~ 




FAiC: 


u 


E4i4eii Address: 

htuskr(»Uf>y^('<cr^ 





ERIC 



(ovei) 



Ill: DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE): 

ff permission to repr^oduce is not granted to ERIC, or, If you wish ERIC to cite the avaiiabiilty of the document from another source, please 
provide the following information regarding the availability of the document. (ERIC will not announce a document unless it is publicly 
available, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria are significantly more 
stringent for documents that cannot be made available through EDRS.) 



Publlsher/Oistri butor: 



Address: 



Price: 



IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER: 

If the right to grant this reproduction release Is held by someone other than the addressee, please provide the appropriate name and 
address: 




V yirLjCDC! ocMr\ 



’HIS FORM: 



Send this form to the following ERIC Clearinghouse: 



However, if solicited by the ERIC Facility, or if making an unsolicited contribution to ERIC, return this form (and the document being 
contributed) to: 

ERIC Processing and Reference Facility 
4483-A Forbes Boulevard 
Lanham, Maryland 20706 



Telephone: 
Toll Free: 
FAX: 



e-mail: 

WWW: 



ERIC 



3 (Rev. 9/971 

>US VERSIONS OF THIS FORM ARE OBSOLETE. 



301-552-4200 

800-799-3742 

301-552-4700 

info@ericfac.piccard.csc.com 

http://ericfacility.org 



